1. Field of the Invention
The present invention relates to distributed computing environments, and deals more particularly with techniques for storing content in these environments to reduce storage overhead and improve efficiency and/or speed of accessing and retrieving the stored content.
2. Description of the Related Art
The popularity of distributed computing networks and network computing has increased tremendously in recent years, due in large part to growing business and consumer use of the public Internet and the subset thereof known as the “World Wide Web” (or simply “web”). Other types of distributed computing networks, such as corporate intranets and extranets, are also increasingly popular. As solutions providers focus on delivering improved web-based computing, many of the solutions which are developed are adaptable to other distributed computing environments. Thus, references herein to the Internet and web are for purposes of illustration and not of limitation.
Millions of people use the Internet on a daily basis, whether for their personal enjoyment or for business purposes or both. As consumers of electronic information and business services, people now have easy access to sources on a global level. Similarly, an enterprise's web-enabled applications may use information and services of other enterprises around the globe. While some content requests are generated programmatically, many content requests have a human user waiting for a response. Returning responses quickly and efficiently can therefore be critical to user satisfaction and to the overall success of an enterprise's web presence. Furthermore, for content types that have a time-sensitive aspect, such as streaming audio and streaming video, processing inefficiencies and network delays must be avoided to the greatest extent possible.
FIG. 1 provides a diagram of a representative server site 100 in which a content request is serviced. (The term “server site” as used herein refers to a collection of server nodes that serve web content associated with a given fully-qualified domain name. For example, the server site 100 in FIG. 1 may, for purposes of example, serve content for a domain name such as “www.ibm.com”.) In this example, a content request 110 is transmitted from a client (not shown) through a network such as the Internet 120 and then to a load balancing host 130 (that is, a computing device which distributes incoming requests across a plurality of web servers 140 to balance the processing load). The load balancing host 130 may then select one of the web servers 140 (such as Apache, Netscape, or Microsoft servers), according to the load balancing strategy which has been implemented in host 130. To serve the requested content, a particular web server may invoke the services of an application server (such as a WebSphere® application server which is available from the International Business Machines Corporation, or “IBM”), where this application server may be co-located with the web server 140 in a single hardware box or may be located at a different device 150. The web server may also or alternatively invoke the services of a back-end enterprise data server 160 (such as an IBM OS/390® server running the DB/2 and CICS® products from IBM), which may in turn access one or more databases 170 or other data repositories. (“WebSphere®”, “OS/390”, and “CICS” are registered trademarks of IBM.) The load balancing host 130 may also function as a surrogate (reverse proxy cache) or intermediary (forward proxy cache). The IBM WebSphere® Edge Server is one implementation which provides this combined functionality. These network components are well known in the art.
As illustrated in FIG. 2, many modern storage systems 200 that may be used by any of the several devices of FIG. 1 are comprised of one or more interconnected physical storage devices 220. Software and/or hardware (not shown) within the storage system 200 function to provide a logical block abstraction layer 210 for the physical storage devices 220; applications using the storage system 200 to read and/or write data then access it via fixed-sized blocks 211.
The physical storage media of devices 220 are typically rotational media (such as disks) accessed by one or more read/write “arms”—that is, sensing devices that can read and write data on the media. Positioning the arms to read/write a particular block of data is a relatively slow operation, and therefore optimizing movement of the arms is a common goal in storage systems. A primary way in which arm movement is optimized is to attempt to locate data such that the physical arm movement can be limited to very short distances. Another common goal in storage systems is to locate data such that, once the arms are in a particular position, the data to be retrieved tends to be contiguous. These data location techniques are often referred to as minimizing the seek time and minimizing the rotational delay, respectively.
It is common for storage systems to optimize block layout on the physical media according to monitored disk access patterns. FIG. 3 is a schematic rendering of a storage system 300 comprised of physical storage devices 220, a usage monitor 310, and a layout optimizer 320. Usage monitor 310 tracks which storage blocks are requested most often. The layout optimizer 320 uses this data to optimize the layout of data on the physical devices 220. To avoid arm movement and rotational delays, blocks that tend to be accessed sequentially can be stored sequentially on the devices 220. The storage system 300 therefore monitors access patterns, and can rearrange the data (if required) to facilitate such sequential read access, thereby reducing the arm movement and rotational delay according to the observed patterns.
In addition, in systems with multiple read/write arms, blocks which tend to be accessed together can be spread among the media serviced by these arms such that the blocks can be accessed simultaneously (or nearly simultaneously). The arm scheduler is then responsible for ensuring that the arms are correctly positioned for the concurrent access operations.
The drawback to the prior art approach is that it relies on observed access patterns. In complex systems, it can be compute intensive and error-prone to detect such patterns. If the observation period is too short, spurious and non-recurring patterns can be detected; if the period is longer, then a larger amount of data is gathered, and must be culled to determine the patterns it represents. This analysis can be very time consuming and compute intensive, resulting in increased system overhead.
What is needed are improved techniques for placing content on storage devices to facilitate efficient content retrieval.